Abstract:

Cognitive diagnostic computerized adaptive testing (CD-CAT) purports to combine the strengths of both CAT and cognitive diagnosis. Currently, most CD-CAT researches focus on the dichotomous data. However there are large polytomous data in educational and psychological tests in practice. This paper proposed a new CD-CAT procedure with polytomous score, called as psCD-CAT. The paper focused on two facets: one was to develop the item selection algorithm, parameter estimation, the polytomous cognitive diagnostic model and terminal rules for psCD-CAT; the other one was to discuss the classification accuracy and item usage in psCD-CAT. Monte Carlo simulation was conducted here. Two experimental conditions were simulated: the fixed test length CAT (10, 15 and 20 items respectively) and the variable test length CAT (the post probability of knowledge state was set to 0.75,0.80 and 0.85 respectively). Simulation studies showed: The classification accuracy, test security and test efficiency under psCD-CAT were all acceptable and reasonable. Whenever under fixed or variable length conditions, PS-PWKL and PS-HKL item selection strategies had high classification accuracy, test security and test efficiency. However, PS-KL item selection strategy had low classification accuracy under fixed variable length conditions and had low test security and test efficiency under variable length conditions. Contract to the random item selection, PS-PWKL and PS-HKL algorithm should be improved to control the item exposure. In conclusion, the proposed psCD-CAT by this paper had overcame the shortcomings stemmed from dichotomously scored CD-CAT, thus they might expect a good prospect and application. And it provided a kind of new methods and techniques in cognitive diagnosis, which might extended the applicable area.

Key words: cognitive diagnosis, CD-CAT, polytomously scored, P-DINA model, GP-DINA model